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(57) Abstract 

The method for adjusting quality during image capture in- 
cludes computing a discrete cosine transform of a digital image to 
create DCT coefficients. A quantization table is generated that 
specifies frequency bands to be filtered and the DCT coefficients 
are digitized using the quantization table. It is preferred that the 
DCT coefficients be ordered in a zig-zag sequence to facilitate 
run-length encoding. 
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QUANTIZATION TABLE ADJUSTMENT 

Background of the Invention 

5 This invention relates to hardware designs coupled with 

software-based algorithms for capture, compression, decompression, and 
playback of digital image sequences, particularly in an editing 
environment 

The idea of taking motion video, digitizing it, compressing the 
10 digital datastream, and storing it on some kind of media for later playback 
is not new. RCA's Sarnoff labs began working on this in the early davs of 
the video disk, seeking to create a digital rather than an analog approach. 
This technology has since become known as Digital Video Interactive 
(DVI). 

15 Another group, led by Phillips in Europe, has also worked on a 

digital motion video approach for a product they call GDI (Compact Disk 
Interactive). Both DVI and CDI seek to store motion video and sound on 
CD-ROM disks for playback in low cost players. In the case of DVI. the 
compression is done in batch mode, and takes a long time, but the 

20 playback hardware is low cost. CDI is less specific about the compression 
approach, and mainly provides a format for the data to be stored on the 
disk. 

A few years ago, a standards-making body known as CCITT, based 
in France, working in conjunction with ISO, the International Standards 

25 Organization, created a working group to focus on image compression. 
This group, called the Joint Photographic Experts Group (JPEG) met for 
many years to determine the most effective way to compress digital 
images. They evaluated a wide range of compression schemes, including 
vector quantization (the technique used by DVI) and DCT (Discrete Cosine 

30 Transform). After exhaustive qualitative tests and careful study, the 
JPEG group picked the DCT approach, and also defined in detail the 
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various ways this approach could be used for image compression. The 
group published a proposed ISO standard that is generally referred to as 
the JPEG standard. This standard is now in its final form, and is 
awaiting ratification by ISO, which is expected. 
5 The JPEG standard has wide implications for image capture and 

storage, image transmission, and image playback. A color photograph can 
be compressed by 10 to 1 with virtually no visible loss of quality. 
Compression of 30 to 1 can be achieved with loss that is so minimal that 
most people cannot see the difference. Compression factors of 100 to 1 and 

10 more can be achieved while maintaining image quality acceptable for a 
wide range of purposes. 

The creation of the JPEG standard has spurred a variety of 
important hardware developments. The DCT algorithm used by the JPEG 
standard is extremely complex. It requires converting an image from the 

15 spatial domain to the frequency domain, the quantization of the various 
frequency components, followed by Huffman coding of the resulting 
components. The conversion from spatial to frequency domain, the 
quantization, and the Huffman coding are all computationally intensive. 
Hardware vendors have responded by building specialized integrated 

20 circuits to implement the JPEG algorithm. 

One vendor, C-Cube of San Jose, California, has created a JPEG 
chip (the CL550B) that not only implements the JPEG standard in 
hardware, but can process an image with a resolution of, for example. 720 
x 488 pixels (CCIRR 601 video standard) in just l/30th of a second. This 

25 means that the JPEG algorithm can be applied to a digitized video 
sequence, and the resulting compressed data can be stored for later 
playback. The same chip can be used to compress or decompress images or 
image sequences. The availability of this JPEG chip has spurred computer 
vendors and system integrators to design new products that incorporate 

30 the JPEG chip for motion video. However, the implementation of the chip 
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in a hardware and software environment capable of processing images 
with a resolution of 640 x 480 pixels or greater at a rate of 30 frames per 
second in an editing environment introduces multiple problems. 
It is often desirable to vary the quality of an image during 
5 compression in order to optimize the degree of data compression. For 

example, during some portions of a sequence, detail may not be important, 
and quality can be sacrificed by compressing, the data to a greater degree. 
Other portions may require greater quality, and hence this greater degree 
of compression may be unsuitable. In prior implementations of the JPEG 
algorithm, quality is adjusted by scaling the elements of a quantization 
table (discussed in detail hereinbelow). If these elements are scaled during 
compression, they must be correspondingly re-scaled during decompression 
in order to obtain a suitable image. This re-scaling is cumbersome to 
implement and can cause delays during playback. The present invention 
is a method that allows for quality changes during compression to enable 
optimum data compression for all portions of a sequence, while allowing 
playback with a single quantization table. 



Summary of the Invention 

20 This invention relates to an apparatus and method for adjusting the 

post decompression quality of a compressed image. The image quality 
adjustment is performed by constructing a quantization table that specifies 
the high frequency image components to be filtered, and by subsequently 
filtering out those components specified by the table. 

25 Brief Description of the Drawing 

Fig. 1 is a block diagram of a video image capture and playback 
system implementing data compression, 

Fig. 2 is a schematic illustration of data compression and 
decompression according to the JPEG algorithm. 
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Descriptioa of the Preferred Embodiment 
A block diagram according to a preferred embodiment of a system 
for capture, compression, storage, decompression, and playback of images 
is illustrated in Fig. 1. 
5 As shown, an image digitizer (frame grabber) 10, captures and 

digitizes the images from an analog source, such as videotape. Image 
digitizer 10 may be, for example, a Truevision NuVista+ board. However, 
the NuVista+ board is preferably modified and augmented with a pixel 
engine as described in copending application "Image Digitizer Including 
10 Pixel Engine" by B. Joshua Rosen et al., filed December 13, 1991, to 

provide better data throughput for a variety of image formats and modes 
of operation. 

The compression processor 12 compresses the data according to a 
compression algorithm. Preferably, this algorithm is the JPEG algorithm. 

15 introduced above. As discussed above, C-Cube produces a compression 
processor (CL550B) based on the JPEG algorithm that is appropriate for 
use as compression processor 12. However, other embodiments are within 
the scope of the invention. Compression processor 12 may be a processor 
that implements the new MPEG (Motion Picture Experts Group) 

20 algorithm, or a processor that implements any of a variety of other image 
compression algorithms known to those skilled in the art. 

The compressed data from the processor 12 is preferably input to a 
compressed data buffer 14 which is interfaced to host computer 16 
connected to disk 18. The compressed data buffer 14 preferably 

25 implements a DMA process in order to absorb speed differences between 
compression processor 12 and disk 18, and further to permit data transfer 
between processor 12 and disk 18 with a single pass through the CPU of 
host computer 16. The host computer 16 may be, for example, an Apple 
Macintosh. 

30 
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JPEG Encoding and Decoding 
Detailed discussions of the JPEG algorithm and its implementation 
are contained in "The JPEG Still Picture Compression Standard" by GK. 
Wallace, in Communications of the ACM, Vol. 34, April 1991, and in 
5 "Digital Compression and Coding of Continuous-Tone Still Images. Part 1, 
Requirements and Guidelines," ISO/IEC JTC1 Committee Draft 10918-1, 
February, 1991, both of which are incorporated herein by reference. 

Fig. 2 illustrates the key steps in data compression and 
decompression according to the JPEG algorithm for a single component of 

10 what will generally be a three-component image. In the JPEG standard, 
an image described in the RGB color space will be transformed into the 
YUV color space via a 3 x 3 multiplier prior to compression. This 
conversion sacrifices some color information, but preserves "the more 
important detail information. 

15 The algorithm works with blocks of 8 x 8 pixels from the image. 

Each 8x8 block is input to the compressor, goes through the illustrated 
steps, and the compressed data is output as a data stream. 

The first step in the JPEG algorithm is a Forward Discrete Cosine 
Transform (FDCT). As described in Wallace, cited above, each 8x8 block 

20 of pixels can be thought of as a 64-point discrete signal which is a function 
of two spatial dimensions. The FDCT computes the "spectrum" of this 
signal in the form of 64 two-dimensional "spatial frequencies/' termed DCT 
coefficients. The DCT coefficients represent the relative amounts of the 
two-dimensional spatial frequencies contained in the 64-point discrete 

25 signal. The coefficient with zero frequency in both dimensions is called the 
"DC coefficient" and the remaining 63 coefficients are called the "AC 
coefficients." Typically each pixel component corresponds to 8 bits, as is 
the case in 24 bit color. According to the JPEG algorithm, each coefficient 
is described by greater than 8 bits. In the C-Cube chip discussed above. 

30 the number of bits per coefficient is 12. Therefore, at this point, the 
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algorithm has actually led to an expansion, rather than a compression of 
data. However, since pixel values usually vary slowly across an image, 
most of the pixel information will be contained in the lower spatial 
frequencies. For typical 8x8 pixel blocks, most of the spatial frequencies 
5 at the high end of the spectrum will have zero or negligible amplitude. 
Data compression can then be achieved by "throwing out" these 
coefficients, which is the purpose of the next step. 

The next step in the JPEG algorithm is quantization, wherein each 
of the 64 DCT coefficients is quantized in accordance with a 64-element 

10 quantization table. This table is specified by the user. The C-Cube chip 
allows user adjustability of this table via software inputs to the chip. Each 
element in the table is any integer from 1 to 255, according to the JPEG 
standard. Each element is the quantizer step size for a corresponding 
DCT coefficient. Quantization is achieved by dividing each DCT coefficient 

15 by its corresponding quantizer step size, and rounding to the nearest 

integer, a very lossy process. The elements of the table are chosen so that 
the generally large lower frequency components are represented by a 
smaller number of bits, and the negligible higher frequency components 
become zero. The goal is to represent each DCT coefficient by no more 

20 precision than is necessary for a desired image quality. Since the 

coefficients, therefore, depend on human visual parameters, the table is 
sometimes called a psycho- visual weighing table. 

Compression is achieved by the use of run-length encoding, which 
puts an end-of-block code at the start of the sequence of zeros that will 

25 typically form the end of the 64 coefficient string. The zeros, therefore, 
don't contribute to the length of the data stream. 

After the coefficients have been quantized, they are ordered into a 
"zig-zag" sequence, as illustrated in Fig. 2. This sequence facilitates the 
run-length encoding. Before going on "to this step, it should be noted, that 

30 since the DC coefficient is generally one of the largest coefficients, and 
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furthermore since it is a measure of the average value of the 64 pixels in 
the 8x8 block, there is generally a strong correlation between the DC 
coefficients of adjacent blocks, and therefore, the DC component is encoded 
as the difference from the DC term of the previous block in the 
5 compression order. 

The final step is entropy coding, wherein additional compression is 
achieved by encoding the quantized DCT coefficients according to their 
statistical characteristics. This is a lossless step. As this step is not as 
relevant to the methods of the present invention as those of the previous 
10 steps, the reader is referred to Wallace, cited above for a detailed 
discussion. 

The above steps are essentially reversed, as illustrated in Fig. lb, 
during playback. Here too, the reader is referred to Wallace for further 
details. 

15 

Image Quality Adjustment 

From the above discussion, it can be seen that image quality can be 
adjusted by scaling the values of the quantization table. For higher 
quality images, the elements should be small, since the larger the 
20 elements, the greater the loss. 

In prior art systems, this is precisely the technique used to adjust 
image quality during image capture. A variable quality scaling factor 
(1-255) called the quantization factor or Q-factor is used with JPEG to 
adjust the degree of quantization of the compressed image. For sequences 
25 requiring high quality, low Q-factors are used. For sequences in which 

quality can be sacrificed, high Q-factors are used. It can be imagined that 
a user may want to continuously adjust the quality over the range of the 
Q-factor at the time of capture as scenes change. 

The problem with the above method is that if the quantization table 
30 values are scaled during image capture, they must be correspondingly 
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descaled during image playback. To illustrate the importance of this, 
imagine the result if the quantization table element corresponding to the 
DC coefficient is multiplied by a factor of 10 at some point during image 
capture in an effort to increase the degree of data compression. If at 
5 playback, the original quantization table is used (prior to the upward 
scaling), the DC coefficient will be 10 times too small. Since the DC 
component primarily corresponds to brightness, the result is dramatic. 

The method of the present invention is an alternate method for 
adjusting quality during image capture which permits playback using a 

10 single quantization table. According to the invention, the DCT coefficients 
are filtered during image capture according to the following technique. 

As has already been discussed, the DC coefficient is the most 
important in terms of human perception. The higher the frequency of a 
coefficient, the finer the detail it describes in an image. Humans are much 

15 less sensitive to these high frequency components. Therefore, according to 
the invention, if image quality is to be lowered to further compress the 
data, the high frequency components are filtered out. The cut-off 
frequency of the filter determines the degree of compression. This method 
is in clear contradistinction to the prior method of adjusting the Q-factor. 

20 As described above and illustrated in Fig. 2, the coefficients are 

sequenced in a zig-zag pattern as part of the quantization step. A filter 
according to one embodiment of the invention can be characterized as a 
diagonal line indicating the cutoff frequency. The effect of throwing out 
the higher frequency components is a blur of the image to an extent 

25 determined by the cutoff frequency. This artifact is often acceptable, 
depending on the scene and the quality required. 

Furthermore, the artifact caused by the filtering can be made more 
tolerable to the eye by adjusting the filter in the following manner. If in 
addition to throwing out all frequency components above cutoff, the 



WO 93/12613 



PCT/LS92/ 10644 



frequency components just below cutoff are muted, the artifact is made 
less harsh. 

The filter described above can be created by hand-creating 
quantization tables. For all frequencies above cutoff, the table elements 
5 should be large, preferably as large as possible without overflowing the 
arithmetic of the system. For frequencies below cutoff, the table elements 
can be exactly as used in standard JPEG implementations. However, 
preferably, the table elements below but near cut-off are increased by some 
amount to mute the corresponding frequency components as described 
10 above. Preferably, this muting is greatest at cutoff, decreasing as the DC 
coefficient is approached. 

The filter can be easily adjusted during image capture to control the 
degree of data compression by changing the quantization table. In one 
mode of operation, the filter is user adjusted. However, in another mode of 
15 operation, the filter may be automatically adjusted by the system when it 
senses bottlenecks forming. 1 

As stated above, the invention was developed as a method for 
adjusting quality during image capture in such a way that playback can 
take place in the absence of the history of such adjustment. It should be 
20 clear that this is achieved when the images are played back using the 
original quantization tables. This is because only the least important 
coefficients are affected by the filtering. In contrast, in the prior methods 
for quality adjustment, all coefficients were affected to the same degree. 

Subsampling introduces artifacts called aliases to the signal. These 
25 frequencies can be predicted and removed by increasing the Q table entries 
for them. 



The interrupt routine gets activated on each frame. It computes the current 
frame size and compares it with the desired target size, then it adjusts the table by 
moving the filter cut-off frequency to approach the target. 
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1. Method for adjusting quality during image capture comprising: 
computing a discrete cosine transform (DCT) of a digital image to 

create DCT coefficients; 

creating a quantization table that specifies frequency bands to be 
filtered; and 

quantizing the DCT coefficients by means of the quantization table 
whereby image quality is adjusted by scaling the values of the 
quantization table. 

2. The method of claim 1 wherein high-frequency components are 
filtered out to further compress the data. 

3. The method of claim 1 further including ordering the quantized 
coefficients in a zig-zag sequence to facilitate run-length encoding. 

4. The method of claim 1 wherein the frequency bands to be filtered 
are user adjusted. 

5. The method of claim 1 wherein the frequency bands to be filtered 
are automatically adjusted. 

6. The method of claim 1 wherein only the least important DCT 
coefficients are affected during the quantizing step. 
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